A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem

We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS occurs naturally in a wide variety of applications where an unknown, non-negative quantity must be recovered from linear measurements. We present a unified framework for S-NNLS based on a rectified power exponential scale mixture prior on the sparse codes. We show that the proposed framework encompasses a large class of S-NNLS algorithms and provide a computationally efficient inference procedure based on multiplicative update rules. Such update rules are convenient for solving large sets of S-NNLS problems simultaneously, which is required in contexts like sparse non-negative matrix factorization (S-NMF). We provide theoretical justification for the proposed approach by showing that the local minima of the objective function being optimized are sparse and the S-NNLS algorithms presented are guaranteed to converge to a set of stationary points of the objective function. We then extend our framework to S-NMF, showing that our framework leads to many well known S-NMF algorithms under specific choices of prior and providing a guarantee that a popular subclass of the proposed algorithms converges to a set of stationary points of the objective function. Finally, we study the performance of the proposed approaches on synthetic and real-world data.Comment: To appear in Signal Processin

Bayesian Sparse Signal Recovery using Scale Mixtures with Applications to Speech

Sparse Signal Recovery (SSR) problem has gained a lot of interest in recent times because of its significant impact on many engineering applications.This thesis tackles this important problem in a Bayesian framework and discusses a generalized scale mixture distribution family, Power Exponential Scale Mixture (PESM) and analyzes its usefulness as a candidate for the sparsity promoting prior distribution. We also derive a unified MAP estimation or Type I framework for SSR by employing an appropriate member of PESM family and show that our unified framework encompasses several popular regularization based SSR algorithms. In addition to that, exploiting natural hierarchical framework induced by the PESM family, we utilize these priors in a Type II/ Empirical Bayes framework and develop corresponding EM based SSR algorithms. Multivariate extension of our proposed PESM family has also been discussed, which in turn resulted in a unified framework for imposing joint sparsity in Multiple Measurement Vector (MMV) recovery problem. We have also shown three specific applications of SSR in audio signal processing, which includes problem specific algorithm enhancements but still utilizes the basic understanding of SSR. For example, by employing a source prior from M-PESM family in a joint blind source separation problem, we propose a realm of reweighted algorithms for Independent Vector Analysis (IVA) with the ability to exploit any intra-source correlation structure. An Empirical bayes based Impulse Response (IR) estimator has also been proposed, which exploits both sparse early reflections and exponential decay reverb tail structure in Room Impulse Response/ Relative Impulse Response as prior information. Sparsity in residual has also been exploited for a speech modeling application, which uses the prior block sparse structure of glottal excitation to find the all pole filter coefficients to model speech efficiently

Bayesian Sparse Signal Recovery using Scale Mixtures with Applications to Speech

Sparse Signal Recovery (SSR) problem has gained a lot of interest in recent times because of its significant impact on many engineering applications.This thesis tackles this important problem in a Bayesian framework and discusses a generalized scale mixture distribution family, Power Exponential Scale Mixture (PESM) and analyzes its usefulness as a candidate for the sparsity promoting prior distribution. We also derive a unified MAP estimation or Type I framework for SSR by employing an appropriate member of PESM family and show that our unified framework encompasses several popular regularization based SSR algorithms. In addition to that, exploiting natural hierarchical framework induced by the PESM family, we utilize these priors in a Type II/ Empirical Bayes framework and develop corresponding EM based SSR algorithms. Multivariate extension of our proposed PESM family has also been discussed, which in turn resulted in a unified framework for imposing joint sparsity in Multiple Measurement Vector (MMV) recovery problem. We have also shown three specific applications of SSR in audio signal processing, which includes problem specific algorithm enhancements but still utilizes the basic understanding of SSR. For example, by employing a source prior from M-PESM family in a joint blind source separation problem, we propose a realm of reweighted algorithms for Independent Vector Analysis (IVA) with the ability to exploit any intra-source correlation structure. An Empirical bayes based Impulse Response (IR) estimator has also been proposed, which exploits both sparse early reflections and exponential decay reverb tail structure in Room Impulse Response/ Relative Impulse Response as prior information. Sparsity in residual has also been exploited for a speech modeling application, which uses the prior block sparse structure of glottal excitation to find the all pole filter coefficients to model speech efficiently